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INTERACTIVE SYSTEM FOR LOCAL 

INTERVENTION INSIDE A 
NONHUMOGENEOUS STRUCTURE 

The invention relates to in interactive system for local 
intervention inside a region of a noohomogcoeous structure. 

The performing of local interventions inside a nonho- 
mogcoeous structure, such as iniracnnial surgical opera- 
tions or orthopedic surgery currently poses the problem of 
optimizing the intervention path or paths so as to secure, on ]0 
the one band, total intervention over the region or structure 
of interest, such as a tumor to be treated or explored and, on 
the other hand, minimal lesion to the regions neighboring or 
adjoining the region of interest, this entailing the localizing 
and then the selecting of the regions of the nonhomogencous 
structure which are least sensitive to being traversed or the IS 
least susceptible to damage as regards the integrity of the 
structure. 

Numerous works aimed at providing a solution to the 
abovcmcnlioned problem have hitherto been the subject of 
publications. Among the latter may be cited the article 20 
entitled "Three Dimensional Digitizer (Ncuronavigator): 
New Equipment for computed Tomography Guided Stereo- 
taxic Surgery", published by Eiju Walanabc, MX)., Takashi 
Watanabe. M.D., Shinya Manaka, M.D., Yoshiaki 
May magi, M.D., and Kin to mo Takakura, M.D. Department 25 
of Neurosurgery, Faculty of Medicine, University of Tokyo, 
Japan, in the journal Surgery Neurol. 1987: 27 pp. 543-547, 
by Elsevier Science Publishing Co., Inc. The Patent WO-A- 
88 09151 teaches a similar item of equipment. 

In the abovementioncd publications are described in 30 
particular a system and an operational mode on the basis of 
which a three-dimensional position marking system, of the 
probe type, makes it possible to mark the three-dimensional 
position coordinates of a nonhomogencous structure, such as 
the head of a patient having to undergo a neurosurgical 35 
intervention, and then to put into correspondence as a 
function of the relative position of the nonhomoexneous 
structure a series of corresponding images consisting of 
two-dimensional images sectioned along ao arbitrary 
direction, and obtained previously with the aid of a medical 40 
imaging method of the "scanner" type. 

The system and the operational mode mentioned above 
offer a sure advantage for the intervening surgeon since the 
latter has available, during the intervention, apart from a 
direct view of the intervention, at least one two-dimensional 45 
sectional view enabling him to be aware, in the sectional 
plane, of the state of performance of the intervention. 

However, and by virtue of the very design of the system 
and of the operational mode mentioned above, the latter 

allow neither a precise representation of the state of perfor- 50 will be given below with reference to the drawings in which: 
mancc of the intervention, nor partially or totally automated FIG. 1 represents a general view of an interactive system 
conduct of the intervention in accordance with a program for for local intervention inside a region of a nonhomogencous 
advance of the instrument determined prior Co the interveo- structure according to the present invention, 
tion. FIG. 2 represents, in the case where the nonhomogencous 

Such a system and such an operational mode cumot 55 Structure consists of the head of a patient, and with a view 
therefore daim to eradicate all man-made risk, since the to a neurosurgical intervention, a reference structure tied to 
intervention is still conducted by the surgeon alone. the nonhomogeneous structure and enabling a correlation to 

The objective of the present invention is to remedy the be established between a "patient" reference frame and a 
whole of the problem cited earlier, and in particular to reference frame of images of the patient which were made 
propose a system permitting as exact as possible a 60 and stored previously, 

correlation, at any instant, between an intervention modeling FIG. 3 represents an advantageous embodiment of the 
on the screen and the actual intervention, and furthermore spacial distribution of the reference structure of FIG. 2, 
the representation from one or more viewing angles, and if FIG. 4 precscnls an advantageous emb o dime nt of the 
appropriate in one or more wrtional planes, of the nonbo- intervention means set up on an operating table in the case 
mogeneous structure, the sectional plane or planes possibly 65 of a neurosurgical intervention, 

being for example pcrpendiculjLr to the direction of the path FIGS. 5a and 56 represent a general flow diagram of 
of advance of the instrument or of the intervention tool functional steps implemented by the system, 



Another objective of the present invention is also the 
implementation of a system permitting simulation of an 
optimal trajectory of advance of the tool, so as to constitute 
an assisted or fully programed intervention. 

Finally, an objective of the present invention is to pro- 
pose a system making it possible, on the basis of the 
simulated trajectory and of the programed intervention, to 
steer the movement of the instrument or tool to the said 
trajectory so as to carry out the programed intervention. 

The invention proposes to this effect an interactive 
system for local intervention inside a region of a nonhomo- 
geneous structure to which is tied a reference structure 
containing a plurality of base points, characterized in that it 
comprises: 

means of dynamic display by three-dimensional imaging 
of a representation of the nonhomogencous structure 
and of a reference structure tied to the nonhomoge- 
neous structure, including images of the base points, 
means of delivering the coordinates of the images of the 

base points in the first reference frame, 
means of securing the position of the non -ho mogeneous 
structure and the reference structure with respect to a 
second reference frame, 
marker means for delivering the coordinates of the base 

points in the second reference frame, 
means of intervention comprising an active member 
whose position b determined with respect to the second 
reference frame, 
means of optimizing the transfer of reference frames from 
the first reference frame to the second reference frame 
and vice versa, on the basis of the coordinates of the 
images of the base points in the first reference frame 
and of the coordinates of the base points in the second 
reference frame, in such a way as to reduce to a 
minimum the deviations between the coordinates of the 
images of the base points in the first reference frame 
and the coordinates of the base points, expressed in the 
said first reference frame with the aid of the said 
reference frame transfer tools, 
means for defining with respect to the first reference 
frame a simulated origin of intervention and a simu- 
lated direction of intervention, and 
reference frame transfer means using the said reference 
frame transfer tools to establish a bidirectional coupling 
between the simulated origin of intervention and the 
simulated direction of intervention and the position of 
the active member. 
A more detailed description of the system of the invention 
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FIGS. 6 thru 8 represent flow diagrams of programs transfer using on the one band the coordinates in » £ 
pcrmiSng impkmcmation of certain fooakmal steps of measured by the probe 3, o a phiraUt^ -of base pomts of die 
pcrmimng '"'P 1 ^ «~ structure SR, and on the other hand the coordinates in R„ 

ric 9a represents a flow diagram of a program permit- determined by graphical tools of the computer MO 1 
ling tapkrneE of a functionalstcp of FTG. 5a, 5 (pomtingbymouse,etc,),of them^of UKC^rKhng 

FIG. 9b represents a flow diagram of a program permit- base points in the representauon RSR, so as to secure die 
Uog LletnTmation of another fractional step of FIG. 5a, best possible correlation between the information modeled 

FIGS. 10a and lto represent a general flow diagram of in the computer equipment and the corresponding real-world 
ihe successive steps of an interactive dialogue between the information. 

system of the present invention and the intervening surgeon 10 There is furthermore provision for reference frarnctrans- 
p fcr means 11 designed lo use the tools thus elaborated and 

* FIG. 10c represents a general flow diagram of the sue- to secure this corrclaUon in realtime, 
cessive functional steps carried out by the system of the Moreover, means 40 are provided, as will be seen in detail 
invention, having (sic) the intervention, prior to the Uter, for determining or modeling a reference origin of 
inlervenlion, during the intervention and after the intcrveo- 15 intervention ORI and a direction of mtervenUon A. 
lion Wilh mc aid of mcans * modeled direction of 

l °liie interactive system for local intervention according to intervention A can, at least prior to the intervention and at the 
the invention will firstly be described in connection wilh start of the intervention, be materialized through an optical 
Pj G j sighting system available to the surgeon. U being possible to 

A nonhomogeneous structure, denoted SNH, on which an 20 steer this sighting system positionally with respect to the 
intervention is to be performed, consists for example of the second reference frame R 2 . 
head of a patient in which a neurosurgical intervention is to The sighting system will be described later, 
be performed. It is however understood that the system of The system of the present - invention finally comprises 
the invention can be used to carry out any type of interven- means 5 of intervention comprising an active member, 
lion in any type of nonhomogencous structure inside which 25 denoted 50, whose position is specified with respect to the 
structural anoVor funclional elements or units may be in second reference frame R 2 . The active member can consist 
evidence and whose integrity, during the intervention, is to of the various tools used in surgical intervention. For 
be respected as far as possible. example, in the case of an .ntercranial neurosurgical 

The system comprises means, denoted 1. of dynamic intervention, the active member could be a trephining tool, 
display by three-dimensional imaging, wilh respect lo a first 30 a needle, a laser or radioscopc emission head, or an endo- 
reference frame R a , of a representation (denoted RSR) of a scopic viewing system. 

reference structure SR (described Uter) tied to the structure According to an advantageous characteristic of the 
SNH and a representation or modeling of the nonhomoge- invention, by virtue of the reference frame transfer means 
neous structure, denoted RSNH. U. ihe position of the active member can be controlled 

More precisely, the means 1 make it possible to dispUy a 35 dynamically on the basis of the pnor modeling of the origin 
plurality of successive three-dimensional images, from dif- of intervention ORI and of the direction of mtervenUon a. 
ferenl angles, of the representations RSNH and RSR. The means 1 of dynamic display by mree^mensional 

The system of the invention also comprises means, imaging of the representations RSNH and RSR comprise a 
denoted 2, of tied positioning, with respect to a second file 10 of two-dimensional image data. Tbc file 10 consists 
reference frame R,. of tbc structures SNH and SR. 40 for example of digitized data from tomographic : sections. 

In the present non-limiting example, the head of the from radiographs, from maps of the patient s head, and 
patient, bearing the reference structure SR, is fixed on an contained in an appropriate mass memory, 
operating table TO to which are fixed the means 2 of tied The successive tomographic sections can be produced 
positioning prior to toe intervention in a conventional manner, after the 

Of course, the patient whose bead has been placed in the 45 reference structure SR has been put in place on the nonho- 
mcans 2 for tied positioning has previously been subjected mogeneous structure SNH. 

to the customary preparations, in order to enable him to According to an advantageous feature, the reference 
undergo the intervention. structure SR can consist of a plurality of marks or notable 

The means2 of the lied positioning wilh respect to Rj will points which can be both sensed by the marker means 3 and 
not be described in detail since they can consist of any so detected on the two-dimensional images obtained, 
means (such as a retaining headset) normally used in the Of course, the abovementioned two-dimensional tomo- 
field of surgery or neurosurgery. The reference frame Rj can graphic sections can likewise be produced by any medical 
arbitrarily be defined as a tri-recUngular reference trihedron imaging means such as a nuclear magnetic resooance sys- 
tied to the operating table TO, as represented in FIG. 1. tern. 

Means 3 of markup with respect to the second reference 55 In a characteristic and well-known maimer, each two- 
frame Rj, tbc coordinates, denoted X2, Y2, Z2, of arbitrary dimensional image corresponding to a tomographic scanner 
points, and in particular of a certain number of base points section corresponds to a structural slice thickness of »bout2 
of the reference structure SR are furthermore provided. to 3 mm, the pixels or image elements in the plane of the 

These base points constituting the reference structure SR tomographic section being obtained with a precision oflne 
can consist of certain notable points andJbc of marks fixed to 60 order of *1 mm. It is therefore understood that the marks or 
the patient, at positions selected by the surgeon and in points constituting the reference structure SR appear on the 
particular at these notable points. images wilh a positional uncertainty, and an important 

The system of the invention fiirtber comprises computing feature of the invention will consist in minimizing ibese 
means 4 receiving means 3 of marking me coordinates X2, uncertainties as wiU be described later. 
X2 t 22. ~ - « Tbc system also comprises first means 110 for calculating 

The computing means 4, as will be seen in detail later, arc and reconstructing three-dimensional images from the data 
designed to elaborate optimal tools for reference frame from the file 10. 



ExhibitJ^ 
Page /l_o\^L 



5,868,675 

5 6 

It also comprises t high -resolution screen 12 permitting determine the coordinates of the said marks in the reference 

the displaying of one or more three-dimensional or two- frame R, or in a reference frame tied to the latter, 

dimensional images constituting so many representations of It is important to note here that the general function of the 

the reference structure RSR and of the nonhomogeoeous base points of the reference structure is, on the one band, to 

structure SNH. * be individually tocalizable on the reference structure, in 

Advantageously, the calculating means 110, the high- order to deduce from this the coordinates in R^, and on the 

resolution screen and the mass memory containing the file other hand, to be visualizable on the two-dimensional 

10 form part of a computer of the workstation type with images so as to be identified (by their coordinates in R,) and 

conventbnal design and denoted MOl. included in the representation RSR on the screen. 

Preferably, the first calculating means 110 can consist of 10 It can therefore involve special marks affixed *t arbitrary 

i CAD type program installed in the workstation MOl. points of the lateral surface of the structure SNH, or else at 

By way of non-limiting example, this program can be notable points of the latter, or else, when the notable points 

derived from the software marketed under tbc tradename can in themselves be localized with high precision both on 

"AUTOCAD" by the "AUTODESK" company in the the structure SNH and on the 2D sections, notable points 

United Slates of America. 15 totally devoid of marks. 

Such software makes it possible, from the various digi- In FIG. 2 a plurality of marks, denoted Ml to Mi, these 

tized two-dimensional images, to reconstruct threcdimen- marks, in the case where the nonhomogeoeous structure 

sional images constituting the representations of the struc- consists of the bead of a patient, being localized for example 

turcs RSR and RSNH in arbitrary orientations. between the eyebrows of the patient, on the lallcr's temples. 

Thus, as has furthermore been represented in FIG. 1, the 20 and at the apex of the skull at a notable point such as the 

calculating means 4 and 11 can consist of a third computer, frontal median point. 

denoted M02 in FIG. 1. * More generally, for a substantially ovoid volume consti- 

Thc first and second computers MOl and M02 are luting tbc nonhomogeoeous structure, there is advanta- 

interconnected by a conventional digital link {bus or geously provision for four base points at least on the outer 

network). 25 surface of the volume. 

As a variant, the computers MOl and M02 can be Thus, as has been represented in FIG. 3, the four marks 

replaced by a single workstation. Ml to M4 of the reference structure are distributed so as 

. The marker means 3 consist of a three-dimensional probe preferably to define a more or less symmetric tetrahedron. 

equipped with a tactile lip 30. The symmetry of the tetrahedron, represented in FIG. 3, is 

This type of three-dimensional probe, known per se and 30 materialized by the vertical symmetry plane PV and ihc 

not described in detail, consists of a plurality of hinged arms, horizontal symmetry plane PH. 

marked in terms of position with respect to a base integral According to an advantageous characteristic, as will be 

with the operating table TO. It makes it possible to ascertain seen later, the means of elaborating the reference frame 

the coordinates of the tactile tip 30 with respect to the origin transfer tools are designed to select three points of the 

O a of the reference frame R 3 with a precision better than 1 35 tetrahedron which will define the "best plane" for the 

mm . reference frame transfer. 

The probe b for example equipped with resolvers dcliv- Also, ihe presence of four or more points enables the 

ering signals representing the instantaneous position of the additional points) to validate a specified selection, 

abovementioned tactile tip 30. The resolvers are themselves More precisely, the presence of a minimum of four base 

connected to the circuits for digital/analog conversion and *o points on the reference structure makes it possible to search 

sampling of the values representing these signals, these for the minimum distortion between the points captured on 

sampling circuits being interconnected in conventional man- the patient by ihe marker means consisting for example of 

ner to the second computer M02 in order to supply it with the three-dimensional probe and the images of these points 

the coordinates X2, X2. 22 of tbc tactile tip 30. on the representation by three-dimensional imaging, the 

As a variant or additionally, and as represented 45 coordinates of which are calculated during processing. The 

diagrammatically, the marker means 3 can comprise a set of best plane of the tetrahedron described earlier, that is to say 

video cameras 31 and 32 (or else infrared cameras) enabling the plane for which the uncertainty in the coordinates of the 

pictures to be taken of the structures SNH and SR. points between the points actually captured by the three- 

The set of cameras can act as a stereoscopic system dimensional probe and the points of the representation of the 

permitting the positional plotting of the base points of the 50 reference structure RSR, is minimal, then becomes the 

reference structure SR, or of other points of toe nonbomo- reference plane for the reference frame transfer. Thus, the 

gencous structure SNH, with respect to the second reference best correlatioo will be established between a modeled 

frame R 2 . Tbc positional plotting can be done for example direction of intervention and a modeled origin of 

by appending a laser beam emission system making it intervention, on the one hand, and the action of the member 

possible to illuminate successively the points whose coor- 55 50. Preferably, the origin of intervention will be placed at the 

dtnatcs are sought, appropriate software making it possible center of Ihe region in which the intervention is to be carried 

to then determine the position of these points one by one out, that is to say a tumor observed or treated for example, 

with respect to Rj. This software will not be described since Furthermore, it will be possible to take the noted residual 

it can consist of posiiioo and shape recognition software uncertainty into account in order to effect the representation 

normally available on the market 60 of the model and of the tools on tbc dynamic display means. 

According to another variant, the marker means 3 can A more detailed description of the means of intervention 

comprise a telemetry system. 5 will now be given in connection with FIG. 4. 

In this case, tbc marks of the structure SR can consist of Preferably, the means of intervention 5 comprise a car- 
small ramotransmitters implanted for example on the rel- riage 52 which is translationally mobile along the operating 
evant points of the patient's head and designed to be visible 65 table TO, for example on a rack, denoted 54, whilst being 
on the two-dimensional images, appropriate electromagnetic driven by a motor, not shown, itself controlled by the 
or optical sensors (not shown) being provided in order to computer M02 for example, via an appropriate link. This 



ExhibitJZL 
Page_/£ofo>Z 



5,868,675 

7 8 

movement system will not be described to deuil since U appropriate, make it possible to record the path of the 
corresponds to a convention!] movement system available sighting direction, constituting in reticular the actual direc- 
oo tbTmaikeL As a variant, the carriage 52 can be mobile tioo of intervention, on the reprcscnlation of the oonhomo- 
"e* a distinct path separated from the opcralmg Ublc TO. ^ = t^ dy^ 
or immobile with respect to the opcratmg table and then 5 J™J^^ 

constitute a support simulated intervention path and steer thereto the move- 

The support carriage 52 comprises in the first pUce a of lhc activc roc[ ^ bcr 50 m ^ nonhomogencous 

sighting member OV. constituting the above-mentioned jn ^ cffcctivcIy to 5^ aU or part of the 

sighting system, which can consist of a binocular telescope. 

The sighting member O V enables the surgeon, prior to the 10 In |hfa ^ of lhc intervention tool 50 is then 

actual intervention, or during the latter, to sight the pre- stcerc d directly to the simulated path (data ORI. A) by 
sumcd position of the region in which the intervention is to involving the reference frame transfer means 11 in order to 
be carried out express the path in the reference frame R 2 . 

Furthermore, and in a non-limiting manner, with the ^ morc <]ctailed description of the implementation of the 
sighting member OV can be associated a hclium-ncon laser 15 operational mode of the system of the invention will now be 
emission system, denoted EL, making it possible to secure described in connection with FIGS. 5a and Sb. 
the aiming of a fine positioning or sighting laser beam on the According to FIG. 5a, the first step consists in obtaining 
structure SNH and in particular, as will be seen in detail and organizing in memory the two-dimensional image data 
later, to indicate to the surgeon the position of an entry point (step 100). Firstly, the nonhomogencous structure SNH is 
PE prior to the intervention, lo enable the latter to open the 20 prepared. In the case of a neurosurgical intervention for 
skull at the appropriate location, and likewise to indicate to example, this means that the patient's head can be equipped 
him what the direction of intervention will be. Additionally, with marks constituting the base points of the reference 
the illuminating of the relevant point of the nonhomogc- structure SR. These marks can be produced by means of 
neous structure or at the very least the lateral surface of the points consisting of a dye partially absorbing the X-rays, 
Utter enables the video cameras 31 and 32 lo carry out, if 25 such as a radiopaque dye. 

necessary, a positional plotting. The abovementioned marks arc implanted by the surgeon 

Preferably, a system for measuring position by telemetry on the patient's head at notable points of the latter [slcJ, and 
53 is provided to secure the precise measurement of the images can then be taken of the nonhomogencous structure 
position of the support carriage 52 of the sighting member SNH by tomography for example, by means of an apparatus 
OV and of the laser emission system EL. During the 30 of the X-ray scanner type. 

operation, and in order to secure the intervention, the This operation will not be described in detail since it 
carriage 52 can be moved along the rack 54, the position of corresponds to conventional operations in the field of mcdi- 
Ihc carriage 52 being measured very precisely by means of cal imaging. 

the system 53. The telemetry system 53 is interconnected The two-dimensional image data obtained are then con- 
with the microcomputer M02 by an appropriate link. 35 stituted as digitized data in the file 10, these data being 

The means of intervention 5 can advantageously consist themselves marked with respect to the reference frame R, 
of a guide arm 51 for the active member 50. and making it possible, on demand, to restore the two- 

The guide arm 51 can advantageously consist of several dimensional images onto the dynamic display means 1, 
hinged segments, each binge being equipped with motors these images representing superimposed sections of the 
and rcsolvers making it possible to secure control of move- 40 nonhomogencous structure SNH. 

ment of the end of the support arm and the positional From the digitized image data available to the surgeon, 
plotting of this same end and therefore of the activc member the latter then proceeds, as indicated at 101 in FIG. 5a, to 
50 according to six degrees of freedom with respect to the select the structures of interest of the abovementioned 
carriage 52. The six degrees of freedom comprise, of course, images. 

three translation^ degrees of freedom with respect lo a 45 The purpose of ibis step is to facilitate ihe work of the 
reference frame tied to the carriage 52 and three rotational surgeon by forming three-dimensional images which con- 
degrees of freedom along these same axes. tain only the contours of the elements of the structure which 

Thus, the support arm 51 and the member 50 arc marked are essential for geometrical definition and real-lime moni- 
in terms of instantaneous position with respect to the second to ring of Ihe progress of ihe intervention, 
reference frame on the one hand by way of the positional so lo the case where the nonhomogeneous structure SNH 
plot of the mobile carriage 52 and, on the other band, by way consists of the head of a patient, an analysis of the t wo- 
of the rcsolvers associated with each binge of the support dimensional image data makes it possible, from values of 
ifiQ 5i. optical density of the corresponding image-points, straigbt- 

In Ihe case of an intracranial neurosurgical surgical away lo extract the contours of the skull, to check the 
intervention, the active member 50 can be removed and can 55 distance scales, etc 

consist of a trephining tool, a needle or radioactive or Preferably, Ihe abovementioned operations arc performed 
chemical implant, a laser or radioisotope emission bead or on a rectangle of interest for a given two-dimensional image, 
an endoscopic viewing system. These various members will this making it possible, by rnoving the rectangle of interest, 
not be described since they correspond to instruments nor- to cover Ihe whole of the image. 

mally used in neurosurgery. 60 The above analysis is performed by means of suitable 

Toe materializing of the modeled direction of intervention software which thus makes it possible lo extract and vec- 
can be effective by means of the laser emitter EL. This torize the contours of the structures which will be modeled 
sighting being performed, the guide arm 51 can then be in the representations RSNH and RSR. 
brought manually or in steered manner into superposition The structures modeled in the case of a neurosurgical 
with ihe direction of intervention A. — 65 intervention are Cor example the skull, the cerebral 

In the case of manual positioning, the rcsolvers associated ventricles, the tumor to be observed or treated, the Calx 
with the sighting member OV and the laser emitter EL, if cerebri, and the various functional regions. 
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According to a feature of Ibc interactive system of the The following step 106 consists in identifying on the 

invention, the surgeon may have available a digitizing table representations RSNl I and RSR displayed on the screen the 

or other graphics peripheral making it possible, for each images of the base points which have just been marked, 

displayed two-dimensional image, to rectify or complete the More precisely, with the aid of appropriate graphics 

definition of the contour of a particular region of interest 5 peripherals, these representations (images) of the base points 

It will be noted finally that by superimposing the extracted tre selected one by one, the workstation supplying on each 

contours on the displayed two-dimensional image, the sur- occasion (in this instance to the computer M02) the coor- 

geon will be able to validate the extractions carried out dinates of these points represented in the reference frame R, . 

The extracted contours are next processed by sampling jbus the computer M02 has available a first set of 
points to obtain their coordinates in the reference frameR,, JQ thrc^imcQsionil coordinates representing the position of 

it being possible to constitute these ^coordinates as an ASCII ^ blsc • u ; Q R tnd , second set of three-dimensional 

type file. This involves step 102 for generating the three- rcprescmmg th c position of the representations 

dimensional data base. , . . - otc • D 

•n,U S .cp isfoHov^by , stop l«^«^|be feswre of ^ these 

three -dimensional model. This step consists firstly, with the .... b . . ,. . ,, Afi 1M , M !.f r 

aid of the CAD type software, in carrying out on the basis " data will be used to elaborate at 108 109, tools for reference 

of the contours ofTbc structures of interest constituted as &"« transfer (from R, to R 2 and vice versa) by calling 

vectorized two-dimensional images an extrapolation upon an intermediate reference frame determined from the 

between the various sectional planes. base points and constituting an intermediate reference frame 

Thc abovementioned extrapolation is carried out prefer- specific to the reconstructed modcL 
ably by means of a "B-spline" type algorithm which seems 20 More precisely, the intermediate reference frame is con- 
best suited. This extrapolation transforms a discrete item of structed from three base points selected so that,- in this 
information, namely the successive sections obtained by reference frame, the coordinates of the other base points 
means of the scanner analysis, into a continuous model after transfer from Rj and the coordinates of thc rcprcscn- 
permitting three-dimensional representation of the volume talions of these other base points after transfer from R, are 
envelopes of the structures. 25 expressed with the greatest consistency and minimum dis- 

It should be noted that the reconstruction of the volumes lortion. 

constituting the structures of interest introduces an approxi- When the step of elaborating the reference frame transfer 

mation related in particular to thc spacing and non-zero tools is concluded, these tools can be used by he system to 

thickness of thc acquisition sections. An important charac- secure optimal coupling between the real world and thc 

tcrislic of thc invention, as explained in detail elsewhere, is 30 computer world (step 1110). 

on the one hand to minimize the resulting uncertainties in the Furthermore, according to a subsidiary feature of the 

patient-model correlation, and on the other band to take into present invention, the system can create on the display 

account the residual uncertainties. means a representation of the nonhomogencous structure 

The CAD type software used possesses standard functions and of the intervention member which lakes account of the 

which enable the model to be manipulated in space by 35 deviations and distortions remaining after the "best" refer- 

displaying it from different viewpoints through just a critc- encc frame transfer tools have been selected (residual 

rion defined by the surgeon (step 104). uncertainties). More precisely, from these deviations can be 

The software can also reconstruct sectional representation deduced by the calculating means a standard error likely to 

planes of the nonbomogeneous structure which differ from appear in the mutual positioning between the representation 

the planes of thc images from the file 10, this making it 40 of the nonhomogencous structure and the representation of 

possible in particular to develop knowledge enhancing the elements (tools, sighting axes, etc.) referenced on R 2 when 

data for the representation by building up a neuro- using the reference frame transfer tools. This residual 

anatomical map. uncertainty, which may in practice be given substance 

The surgeon can next (step 105) determine a model of through an error matrix, can be used for example to rcprc- 

intervention strategy taking into account the modeled struc- 45 sent certain contours (tool, structures of interest to be 

tures of interest, by evaluating the distance and angle ratios avoided during the intervention, etc.) with dimensions larger 

00 the two-and three-dimensional representations displayed. than those which would normally be represented starting 

This intervention strategy will consist, in actual fact, on from the three-dimensional data base or with the aid of 

the one band in localizing the tumor and in associating coordinates marked in Rj, the said larger dimensions being 

therewith a "target point", which will subsequently be able 50 deduced from the "normal" dimensions by involving the 

to substitute for the origin common to all the objects (real error matrix. For example, if the member were represented 

and images) treated by the system, and 00 the other hand in normally, in transverse section, by a circle of diameter Dl, 

determining a simulated intervention path respecting to the a circle of diameter D2>D1 can be represented in substance, 

maximum the integrity of the structures of interest This step with the difference D2-D1 deduced from the standard error 

can be carried out "in the office", involving only the work- 55 value. In this way, when a direction of intervention will be 

station. selected making it possible to avoid traversing certain struc- 

Once this operation is performed and prior to the tures of interest, the taking into account of an -enlarged" size 

intervention, the following phase consists in implementing of the intervention tool will eradicate any risk of the 

the steps required to establish as exact as possible a cone- member, because of the abovementioned errors, accidently 

lation between the structure SNH (real world) and the 60 traversing these structures. 

representation RSNH (computer world). This involves steps Back at step 105, and as will be seen in more detail with 

106 to 109 of FIG. 56. reference to FIGS. 9a and 96, the reference origin of 
Firstly, as represented in FIG. 56 at step 107, marking of intervention ORI and the direction of intervention A, that is 
the base points of the reference structure SR with respect to to say the simulated intervention path, can be determined 
the second reference frame is carried out with the aid of the 65 according to various procedures. 

marker means 3, by delivering to the system the coordinates According to a first procedure, the trajectory can be 

X2, Y2, Z2 of the said base points. defined from two points, namely an entry point PE (FIG. 3) 
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and a Uiget point, thil is to siy substantially the center of the 
structure of interest consisting of the tumor to be observed 
or treated. Initially, these two points axe localized on the 
>- model represented on the screen. 

According to a second methodology, the trajectory can be 
determined from the above mentioned target point and from 
a direction which takes account of the types of structures of 
interest and of their positions with a view to optimally 
respecting their integrity. 

After the above mentioned step 108, the surgeon can at 
step 1110 perform the actual intervention. 

The intervention can advantageously be performed by 
steering the tool or active member over the simulated 
intervention path, determined in step 1110. 

As a variant, given that the support arm 51 for the active 
member, equipped with its rcsolvers, continuously delivers 
the coordinates in R 2 of the said active member to the 
system, it is also possible to perform the operation manually 
or semi-manually, by monitoring on the screen the position 
and motions of a representation of the tool and by comparing 
them with the simulated, displayed intervention path. 

It will furthermore be noted that the modeled direction of 
intervention can be materialized with the aid of the laser 
beam described earlier, the positioning of the latter (with 
respect to R^ being likewise carried out by virtue of (he 
reference frame transfer tools. 

Certain functional features of the system of the invention 
will now be described in further detail with reference to 
FIGS. 6, 7, 8, 9a and 96. 

The module for elaborating the reference frame transfer 
tools (steps 108, 109 of FIG. 5b) will firstly be described 
with reference to FIG. 6. 

This module comprises a first sub-module 1001 for 
acquiring three points A, B,C, the images of the base points 
of SR on the representation RSNH (the coordinates of these 
points being expressed in the computer reference frame R,), 
by successive selections of these points on the representa- 
tion. To this effect, the surgeon is led, by means of a graphics 
interface such as a "mouse" to point successively at the three 
selected points A, B, C 

The module for preparing the transfer tools also com- 
prises a second sub-module, denoted 1002, for creating a 
unit three-dimensional orthogonal matrix M, this matrix 
being characteristic of a right-banded orthonormal basis 

represented by three unit vectors i , j , k , which define an 
intermediate reference frame tied to R,. 

The unit vectors T, X and £ are given by the relations: 




where Q Q designates the norm of the relevant vector. 

In the above relations, the sign "A" designates the vector 
product of the relevant vectors. 

Similarly, the module for preparing the transfer tools 
comprises a third sub-cnodule, denoted 1003, for acquiring 
three base points D, E, F, of the structure SR, these three 
points being those whose images on the model are the pouts 
A, B, C respectively. For this purpose, the surgeon, for 
example by means of the tactile tip 30, successively senses 
these three points to obtain their coordinates in R^. 
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The sub-module 1003 is itself followed, as represented in 
FIG. 6, by a fourth sub-module 1004 for creating a unit 
tbixc-dimensional orthogonal matrix N, characteristic of a 
right-handed orthonormal basis comprising three unit vec- 

5 tors T\ 7* IT* and which is tied to the second reference 
frame R 2 owing to the fact that the nonhomogeoeous struc- 
ture SNH is poslionatly tied with respect to this reference 
frame. 

The three unit vectors i\ j \ k' arc defined by the 
10 relations: 



15 




/"A ** 



20 As indicated above, to the extent that the base points of 
the reference structure can be marked in R 2 with high 
precision, so their representation in the computer base Rj is 
marked with a certain margin of error given on the one hand 
the non-zero thickness (typically from 2 to 3 mm) of the 

25 slices represented by the two-dimensional images from the 
file 10, and on the other hand (in general to a lesser extent) 
the definition of each image element or pixel of a section. 

According to the invention, once a pair of transfer matri- 
ces M, N has been elaborated with selected points A, B, C, 

30 D, E, F, it is sought to validate this selection by using one or 
more additional base points; more precisely, for the or each 
additional base point, this point is marked in R 2 with the aid 
of the probe 30, the representation of this point is marked in 
R, after selection on the screen, and then the matrices N and 

3J M are applied respectively to the coordinates obtained, in 

order to obtain their expressions in the bases ( i ', J\ k *) and 

(7, j , k) respectively. If these expressions are in good 
agreement, these two bases can be regarded as a single 
intermediate reference frame, this securing the exact as 

40 possible mathematical coupling between the computer ref- 
erence frame Rj tied to the model and the "real" reference 
frame R a tied to the patient. 

In practice, the module for elaborating the reference frame 
transfer tools can be designed to perform steps 1001 to 1004 

45 in succession on basic triples which differ on each occasion 
(for example, if four base points have been defined associ- 
ated with four representations in RSR, there are four pos- 
sible triples), in order to perform the validation step 1005 for 
each of these selections and finally in order to choose the 

50 triple for which the best validation is obtained, that is to say 
for which the deviation between the above mentioned 
expressions is smallest This triple defines the "best plane" 
mentioned elsewhere in the description, and results in the 
"best" transfer matrices M and N. 

55 As a variant, it WO] be possible for the selection of the best 
plane to be made at least in part by the surgeon by virtue of 
his experience. 

It should be noted that the reference frame transfer will 
only be concluded by supplementing the matrix calculation 

60 with the matrices M, N with a transfer of origin, so as to 
create a new common origin for example at the center of the 
tumor to be observed or treated (point ORI). This transfer of 
origin is rffrrlfd simply by appropriate subtraction of vec- 
tors 00 the one hand on the coordinates in R„ and on the 

65 other hand on the coordinates in These vectors to be 
subtracted arc determined after localization of the center of 
the tumor on the representation. 
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Furthermore, the means described above for establishing From a simulated intervention path, for example on the 

the coupling between the patient's world and the model's reprosecution RSNH, or from any other straight line 

world can also be used to couple to the model's world thai selected by the surgeon, the invention furthermore enables 

of map data* also stored in the workstation and expressed in me m^jej to be represented on the screen from a viewpoint 

a different reference frame denoted R 3 . In this case, since 5 corTtS p OOC ^ n g to this straight line. Thus the third transfer 

these data contain no specific visible mark, the earlier comprises, as represented in FIGS. 9a and 9b. a 

^scribed dabon^ of matrices is ^^^^ first module 301 for visualizing the representation in a 

ine for these marks the positions of notable points ol the 7. . . . . . j . , r „, 

paUem's head. These m.Ybe temporal points/the frontal *™*<* bv *"» P 0 ** « d 1 ^ for 

median point, the apex of the skull, the center of gravity of visualizing the representation in a direction given by an 

the orbits of the eyes. etc. ™gle of elevation and an angle of azimuth. 

The corresponding points of the model can be obtained The first module 301 for visualizing the representation in 

cither by selection by mouse or graphics tablet on the model, i direction given by two points comprises a first sub-module 

or by sensing on the patient himself and then using the denoted 3010 permitting acquisition of the two relevant 

transfer matrices. points which will define the selected direction. The coordi- 

The above step of elaborating (he reference frame transfer Qalcs of mcsc . ^ &rc exprcssed - m mc reference frame R„ 

tools, conducted in practice ^ by the calculating means 4, ^ ^ uircd prcviousIy O0 lhc 

makes it possible subsequently to implement the reference . ^ , CW]1 7 . . ' .„„ rtf 

f me transfermcans (FIGS 7 and 8) nonhomogeneous structure SNH for example by means of 

"with refcrena to FIG. 7. the first transfer sub-module 201 t"*ilc *9 30 and then subjected to the reference frame 

comprises a procedure denoted 2010 for aquiring the coor- 20 transfer, or chosen directly on the representation by means 
dinates XM, YM, ZM, expressed in R„ of the point to be . of the graphics interface of the ''mouse" type, 

transferred, by selecting on the representation. The first sub-module 3010 is followed by a second 

The procedure 2010 followed by a procedure 2Q11 .for sub-module denoted 3011 permitting the creation of a unit, 

calculating the coordinates XP, YP, ZP (expressed in Rj) of orthogonal three-dimensional matrix V characteristic of a 

the corresponding real point on the patient through the 25 rf ht . haodcd Ofthooormal basis 7\ ]\ k" the unit vectors 

transformation: _ _ 

{XP. YP. ZPJ-M'N-^JXM. YM, ZM} where M * N* 1 i \ j k', being determined through the relations: 
represents the product of the matrix M and the inverse 

matrix N. — — — 

The procedure 2011 is followed by a processing proce- 30 *" * Afi/tlA5 * 

dure 2012 utilizing the calculated coordinates XP. YP, ZP, 7* ** - 0,7" *"? - Oft f I - 1; 

for example to indicate the corresponding point on the - - 

surface of the structure SNH by means of the laser emission ** ' 
system EL> or again to secure the intervention at the relevant 

point with coordinates XP, YP, ZP (by steering the active 35 where **A" represents the vector product and symbolizes 

member). the scalar product. 

Conversely, in order to secure a transfer from SNH to sub-module 3011 is followed by a routine 3012 

RSNH, the second sub-module 202 comprises (FIG. 8) a maJdng it possible to secure for all the points of the entities 

procedure denoted 2020 for acquiring on the structure SNH (structures of interest) of the three-dimensional data base of 

the coordinates XP,YP,ZP (expressed in Rj) of a point to be 40 coordinates XW, YW, ZW in R, a conversion into the 

transferred. , . . . -r*. 7*% , , . 

These coordinates can be obtained by means of the tactile orlbonormal baas ( 1 \ j , k ) by the relation: 

lip 30 for example. The procedure 2020 is followed by a {XV, YV, ZV}-V{XW, YW. ZW) 

procedure 2021 for calculating the corresponding coordi- The subroutine 3013 is then followed by a subroutine 

nates XM, YM, ZM in R, through the transformation: 45 3014 for displaying the plane i\ j\ the subroutines 3013 and 

{XM. YM, ZM}-N*M" 1# {XP, YP, ZP} 3014 being called up for all the points, as symbolized by the 

A procedure 2022 next makes it possible to effect the arrow returning to block 3012 in FIG. 9a. 

displaying of the point with coordinates XM. YM, ZM on When all the points have been processed, an output 

the model or again of a straight line or of a plane passing module 3015 permits return to a general module, which will 

through this point and furthermore meeting other criteria. 50 be described later in the description. It is understood that this 

It will be noted here that the two sub-modules 201, 202 module enables two-dimensional images to be reconstructed 

can used [sic] by the surgeon at any moment for the purpose in planes perpendicular to the direction defined by A and B. 

of checking the valid nature of the transfer tools; in In the same way, the second module 302 (FIG. 9b) for 

particular, it is possible to check at any time that a real base visualizing the representation from a viewpoint given by an 

point, with coordinates known both in R, and R a (for 55 angle of elevation and an angle of azimuth comprises a first 

example a base point of SR or an arbitrary notable point of sub-module 3020 for acquiring the two angles in the repre- 

the structure SNH visible on the images), correctly relocates sentation frame of reference. 

with respect to its image after transferring the coordinates in The selectioo of the angles of elevation and of azimuth 

step 2011. can be made by selecting from a predefined data base or by 

In the event of an excessive difference, a new step of 60 moving software curse rs associated with each view or else 

elaboration of the transfer tools is performed. by modification relative to a current direction, such as the 

Furthermore, the sub-modules 201, 202 can be designed modeled direction of intervention. Tbc sub-module 3020 is 

to also integrate the taking into account of the residual itself followed by a second sub-module 3021 for creating a 

uncertainty, as spoken of above, so as for example to unit orthoginil ihrcr-d matrix W rh«rarirri sticof 

represent : <m the ; screen a point sensed^ in a pointwise 65 basis 0 f unit vectors T«, 
manner, but m the form for example of a circle or a sphere 

representing the said uncertainty. k*\ They are defined by the relations: 
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7--I--0; 

7-T--0. 

7" • 7 • cos(elcvtiion); 
I " * x " - sin(elev»tioa) 

7* - X** a 7* 

A roulinc 3022 is Ibca called for all tbe points of the 
coiities of the three-dime osiooal data base of coordinates 
XW, YW, ZW and enables a first sub-routine 3023 to be 
called permitting calculation of the coordinates of the rel- 
evant point in tbe right-handed ort bo normal bases i '\ 

J" through the transformation: 

{XV, YV, ZV}-V{XW, YW, ZW} 

The sub-routine 3023 is itself followed by a sub-routine 
3024 for displaying the plane f\ j*", the two sub-routines 
3023 and 3024 then being called up for each point as 
symbolized by the return via the arrow to tbe block 3022 for 
calling the abovemcnlioncd routine. When all the points 



Another screen page 4010 is then provided, so as to effect 
the localizing of the target on the representation (for 
example' a tumor to be observed or treated in the case of t 
ocurosurgical intervention) in order subsequently to deter- 
5 mine a simulated intervention path. 

Then a new screen page 4011 makes it possible to effect 
tbe setting of the guides for the tool on the basts of this 
simulated path before opening up tbe skin and bone flaps on 
the patient's skull. ' 
Then a new localizing step 4012 makes it possible to 
10 check whether the position of tbe guides corresponds cor- 
rectly to the simulated intervention path. 

The screen page 4012 is followed by a so-called inter- 
vention screen page, the intervention being performed in 
accordance with step 1110 of FIG. 56. 
* 5 A more detailed description of the interactive diatogue 
between the surgeon and the system during a surgical, and 
in particular a neurosurgical, intervention will follow with 
reference to FIG. 10c and to all of the preceding description. 
The steps of FIG. 10c arc also integrated in the general 
20 program mentioned earlier; there arc undertaken in succes- 
sion a first phase I (preparation of the intervention), then a 
second phase II, (prior to tbe actual intervention, the patient 
is placed in a condition for intervention, the reference 
structure SR being tied to the second reference frame Rj), 



return to tbe general menu. 

Of course, all of the programs, sub-routines, modules, 
sub-modules and routines destroyed earlier are managed by 
a general "menu" type program so as to permit interactive 
driving of the system by screen dialogue with the interven- M 
ing surgeon by specific screen pages. 

A more specific description of a general flow diagram 
illustrating this general program will now be given in 
connection with FIGS. 10a and 106. 

Thus, in FIG. 10a has been represented in succession a 
screen page 4000 relating to the loading of data from the 
digitized file 10, followed by a screen page 4001 making it 
possible to secure tbe parameterizing of the grey scales of 
tbe display on the dynamic display means 1 and to calibrate 
the image, for example, 



intervention phase IV. 

With a view to preparing the intervention, tbe system 
requests the surgeon (step 5000) to choose the elementary 
structures of interest (for example bones of the skull, 
ventricles, vascular regions, the tumor to be explored or 
treated, and the images of the marks constituting in the first 
reference frame the representation RSR). 

The choice of the elementary structures of interest is made 
on the display of tbe tomographic images, for example, 
called up from the digitized file 10. 

The system next performs, at step 5001, a modeling of the 
structures of interest, as described earlier. Then, the nonho- 
mogencous structure having been thus constituted as a 
three-dimensional model RSNH displayed on the screen, the 
intervening surgeon is then led to perform a simulation by 



Tbe screen page 4001 is followed by a screen page 4002 « three-dimensional imaging, at step 5002, with a view to 
making it possible to effect the generation of a global view defining the intervention path of the tool 50. 
and then a step or screen page 4003 makes it possible to During phase II the patient being placed in a condition for 
effect an automatic distribution of the sections on the screen intervention and his bead and the reference structure SR 
of the workstation. being lied to the second reference frame R 7 , the surgeon 
A screen page 4004 makes it possible to effect a manual 45 performs at step 5003 a search for the position of the marks 
selection of sections and then a screen page 4005 makes it Ml to M4 constituting base points of the reference structure 
possible to effect tbe selection of the strategy (search for the in the second reference frame Rj, and then during a step 
entry points and for the possible directions of intervention, 5004, performs a search for the position of the sighting 
first localizing of the target (tumor . . . ) to be treated . . . ), systems, visualizing member OV, or of the tools and inter- 
as defined earlier, and to select tbe position and horizontal, 50 vention instruments 50, still in the second reference frame 
sagittal and frontal distribution of the sections. Rj, so as, if appropriate, to align these implements with 

A screen page 4006 also makes it possible to effect a respect to R^. 

display of tbe settings of a possible stereotaxic frame. Tbe system then performs the validation of the 

It will be recalled that the reference structure SR advan- intervention/patient spaces and representation by three- 

tageously replaces tbe stereotaxic frame formerly used to 55 dimensional imaging in order to determine next tbe common 

effect tbe marking of position inside the patient's skulL origin of intervention OR1. In other words, tbe matrix 

There may furthermore be provided a screen-page 4007 reference frame transfer described above is supplemented 
for choosing strategic sections by three-dimensional 
viewing, on selection by the surgeon, and then at 4008 the 
aligning of the references of the peripherals (tool, sighting 60 
members, etc., with the aid of the probe 30. 

A screen page 4009 is also provided to effect tbe search 
for the base points on the patent with the aid of the said 
probe, following which the steps of construction of tbe 

reference frame transfer tools and of actual reference frame 65 (and if appropriate of tbe laser beam), with the direction of 

transfer are performed, preferably in a user-transparent aim (of observation) simulated by three-dimensional imag- 
ing on the display means 1, and vice versa. 



with the necessary origin translations (origins 01 and 02 
aligned on ORI). 

This operation is performed as described earlier. 

Phase III corresponds to tbe intervention, during which 
the system effects at step 5006 a permanent coupling in real 
time between tbe direction of aim of the active member 50, 
and/or of the direction of aim of tbe *»gftting member OV 
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Id the following step 5007, the coupling is effected of toe 
movements and motions of the intervention instrument with 
their movements simulated by ihree-dimcnsiooal imaging, 
with automatic or manual conduct of the intervention. 

As noted at 5008, the surgeon can be supplied with a 
pennaoent display of the original two-dimensional sectional 
images in planes specified with respect to the origin ORI and 
to the direction of intervention. Such a display enables the 
surgeon at any time to follow the progress of the intervention 
in real time and to be assured that the intervention is 
proceeding in accordance with the simulated intervention. In 
phase IV which is executed after the intervention, the system 
effects a saving of the data acquired during the intervention, 
this saving making it possible subsequently to effect a 



10 



external parts of the patient can be visualized as com* 
pared with the capture by the scanner, 

it being possible for the surgeon's field of view given by 
a sighting member (such as a surgical microscope) to be 
referenced with respect to the direction of visual* 
izatioo of the model on the screen can be made identical 
to the real sight by the sighting member; 

finally, the three-dimensional images, normally displayed 
on the screen in the preceding description, may as a 
variant be introduced into the surgeon's microscope so 
as to obtain the superposition of the real image and the 
representation of the model. 

We claim: 

1. An interactive system for local intervention inside a 



comparison in real lime or deferred in the event of succcs- 15 re gi on G f t oon-homogeneous structure to which is con- 



sivc interventions on the same patient. 

Furthermore, the saved data make it possible to effect a 
playback of the operations carried out with the option of 
detailing and supplementing the regions traversed by the 
active member 50. 20 

Thus, a particularly powerful interactive system for local 
intervention has been described. 

Thus, the system which is the subject of the present 
invention makes it possible to represent a model containing 
only the essential structures of the nonhomogencous 25 
structure, this facilitating the work of preparation and of 
monitoring of the intervention by the surgeon. 

Moreover, the system, by virtue of the algorithms used 
and in particular by minimizing the distortion between the 
real base points and their images in the 2D sections or the 30 
maps, makes it possible to establish a two-way coupling 
between the real world and the computer world through 
which the transfer errors arc minimized, making possible 
concrete exploitation of the imaging data in order to steer the 
intervention tool. 35 

To summarize, the system makes possible an be r active 
[sic] medical usage not only to create a three-dimensional 
model of the nonhomogencous structure but also to permit 
a marking in real time with respect to the internal structures 
and to guide the surgeon in the intervention phase. 40 

More generally, the invention makes it possible to cod up 
with a coherent system in respect of: 

the two-dimensional imaging data (scanner sections, 
maps, etc.) 

the three-dimensional data base; 45 
the data supplied by the marker means 3 in the reference 
frame R 2 ; 

the coordinate data for the sighting systems and interven- 
tion tools; M 

the real world of the patient on the operating table. 

Accordingly, the options offered by the system are, in a 
non-limiting manner, the following: 

the tools and of [sic] their position can be represented on 
the screen; 55 

the position of a point on the screen can be materialized 
on the patient for example with the aid of the laser 
emission device EL; 

the orientation and the path of a tool such as a needle can 
be represented on the screen and materialized on the 60 
patient optically (laser emission) or mechanically 
(positioning of the guide-arm in which the tool is 
guided in translation): 

an image of the patient, yielded tor example by a system 
for taking pictures if appropriate in relief, can be 65 
superimposed on the three-dimensional representation 
modeled on the screen; thus, any change in the soft 



occtcd a reference structure containing a plurality of base 
points, the interactive system comprising: 
means for dynamically displaying a three-dimensional 
image of a representation of the non •homogeneous 
structure and of the reference structure connected to the 
non-homogeneous structure, wherein the three- 
dimensional image also includes a plurality of images 
of the plurality of base points; 
means for determining a set of coordinates of the plurality 
of images of the plurality of base points in a first 
reference frame; 
means for fixing a position of the non -homogeneous 
structure and of the reference structure with respect to 
a second reference frame; 
means for determining a set of coordinates of the plurality 

of base points in the second reference frame; 
means of intervention comprising an active member 
whose position is determined with respect to the second 
reference frame; 
means for generating a plurality of reference frame trans- 
lation tools for translating a plurality of reference 
frames from the first reference frame to the second 
reference frame and vice versa, based on the set of 
coordinates of the plurality of images of the plurality of 
base points in the first reference frame and of the set of 
coordinates of the plurality of base points in the second 
reference frame, in such a way as to reduce to a 
minimum at least one of a set of deviations between the 
set of coordinates of the plurality of images of the 
plurality of base points in the first reference frame and 
the set of coordinates of the base points, expressed in 
the first reference frame using the plurality of reference 
frame translation tools; 
means for defining, with respect to the first reference 
frame, a simulated origin of intervention and a simu- 
lated direction of intervention; and, 
means for transferring the plurality of reference frames 
using the plurality of reference frame translation tools 
to *«««M'<*i a bidirectional coupling between the simu- 
lated origin of intervention and the simulated direction 
of intervention and the position of the active member. 
2. The interactive system according to claim 1, wherein 
the plurality of reference frame translation tools comprise: 
means for creating a matrix (M) for transferring between 
the first reference frame and a first intermediate refer- 
ence frame based on a set of coordinates of a set of 
three images of a set of three base points of the 
reference structure; 
means for creating a matrix (N) for transferring between 
the second reference frame and a second intermediate 
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reference frame based on ibe set of coordinates of the 
set of three images of the set of three base points of the 
reference structure; and, 
means for validating matrix (M) and matrix (N) based on 
(he set of three base points and the set of three images, 
such that at least one deviation between an expression 
for at least one additional base point in the second 
intermediate reference frame and an expression for at 
least one image point of the additional base point in the 
first intermediate reference frame is reduced to a mini- 
mum. 

3. Tbc interactive system according to plurality of claim 
2, wherein the means for transferring the reference frames 
using the plurality of reference frame translation tools 
further comprises: 
a first transfer sub-module for transferring a set of 
represenlation/non-homogeneous structure 
coordinates, and 



a file containing digitized data from a set of two- 
dimensional images constituted by successive non- 
invasive tomographic sections of the non- 
bomogeoeous structure; 

means for calculating and reconstructing the three- 
dimensional image from the set of two-dimensional 
images; and 

a high-resolution display screen. 

8. The interactive system according to claim 7, wherein 
the means for calculating and reconstructing the three- 
dimensional image from the set of two-dimensional images 
comprises a program consisting of computer-aided design 
type software. 

9. Tbc interactive system according to claim 1, wherein 
the means for determining the set of coordinates of the 
plurality of base points in the second reference frame 
comprises a three-dimensional probe equipped with a tactile 



a second transfer sub-module for transferring a set of „ * '** f coordinates of the tactile tip in the 

- k c „ r „.,,./r M . V n.,i; n . ™™h;. 20 sari second reference frame. 

10. The interactive system according to claim 1, wherein 
the means for determining the set of coordinates of the 
plurality of base points is the second reference frame com- 
prises at least one of a set of optical sensors and a set of 
electromagnetic sensors. 

11. The interactive system according to claim 1, wherein 
a portion of the set of the plurality of base points of the 
reference structure comprises a plurality of marks positioned 



non -homogeneous structure/representation coord i 
nates. 

4. The interactive system according to claim 3, wherein 
die first transfer sub-module comprises: 

means for acquiring a set of coordinates (XM, YM, ZM), 25 
expressed in the first reference frame, of a point of the 
representation of the non -homogeneous structure to be 
transferred, by selection on the representation; 

means for calculating a set of corresponding coordinates 



(XP. YP, ZP). expressed in the second reference frame, 30 °° > btcn\ surface of the non-bomogeneous structure. 



on the non-homogeneous structure through a transfor- 
mation: 

{YP.YP, ZPl-MTT 1 *{XM,YMZM} where M • W 1 
represents a product of the matrix (M) and an inverse 
of the matrix (N), and 

means for processing, with the aid of the corresponding 
coordinates (YP, YP, ZP), to display a corresponding 
point on a surface of the non-homogeneous structure 
and to secure the intervention. 

5. The interactive system according to claim 3, wherein 
the second transfer sub-module comprises: 

means for acquiring a set of coordinates (XP, YP, ZP), 
expressed in the second reference frame, of a point of 
the non-homogeneous structure to be transferred; 

means for calculating a set of corresponding coordinates 
(XM YM, ZM), expressed in the first reference frame, 
of the representation through a transformation: 

{YM, YM. ZM}-N*M* J *{XP,ZP r ZP} where N*M _1 
represents the product of the matrix (N) and an inverse 50 
of the matrix (M); and, 

means for displaying the representation using the set of 
corresponding coordinates (YM, YM, ZM). 

6. The interactive system according to claim 1, wherein 
the means for generating the plurality of refer en ce frame 55 
translation tools also generate, in association with the ref- 
erence frame translation tools, tools for taking into account 
a residual uncertainty which is based on the set of deviations 
between the set of coordinates of the plurality of images of 



12. The interactive system according to claim 11, wherein 
the plurality of marks are four in number and arc distributed 
over the lateral surface so as to define a substantially 
symmetrical tetrahedron. 
35 13. The interactive system according to daim 1 , wherein 
the means of intervention comprises: 

a guide arm to secure intervention in the region of the 
non-homogeneous structure, the guide arm having a 
position marked with respect to the second reference 
40 frame; and, 

an active intervention member whose position is marked 
with respect to the second reference frame. 

14. Tbc interactive system according to claim 13, wherein 
45 the active intervention member is removable and selected 

from the group consisting of: 
tools for trephining; 
needles and implants; 

laser and radioisotope emission heads; and, sighting and 
viewing systems. 

15. Tbc interactive system according to daim 1, wherein 
the means for transferring the plurality of reference frames 
g<f«hb'A*s a coupling between a direction of visualization of 
the representation of the non-homogcocous structure on the 
display means and a direction of observation of the non- 
homogeneous structure and of the reference structure by the 
active intervention member. 

16. The interactive system according to daim 15, further 



the plurality of base points in the first reference frame and 60 caD P mm & 



the set of coordinates of the base points, the tools for taking 
into account the residual uncertainty usable for displaying a 
set of contours in the representation whilst taking into 
account the residual uncertainties. 

7. The interactive system according to claim 1, wherein 65 
the means of dynamic displaying the three-dimensional 
image comprises: 



a first module for visualizing a representation in a direc- 
tion given by two points; 
a second module for visualizing a representation in a 
direction given by an angle of elevation and an angle of 
azimuth, 





17. An interactive system for local intervention inside a region of a non- 
homogeneous structure to which is connected a reference structure containing a 
plurality of base points, the interactive system comprising: 

means for dynamically displaying a three-dimensional image of a representation 
of the non-homogeneous structure and of the reference structure connected to the non- 
homogeneous structure, wherein the three-dimensional image also includes a plurality 
of images of the plurality of base points; 

means for determining a set of coordinates of the plurality of images of the 
plurality of base points in a first reference frame; 

means for fixing a position of the non-homogeneous structure and of the 
reference structure with respect to a second reference frame; 

means for determining a set of coordinates of the plurality of base points in the 
second reference frame; and 

means for generating a plurality of reference frame translation tools for 
translating a plurality of reference frames from the first reference frame to the second 
reference frame and vice versa, based on the set of coordinates of the plurality of 
images of the plurality of base points in the first reference frame and of the set of 
coordinates of the plurality of base points in the second reference frame, in such a way 
as to reduce to a minimum at least one of a set of deviations between the set of 
coordinates of the plurality of images of the plurality of base points in the first reference 
frame and the set of coordinates of the base points, expressed in the first reference 
frame using the plurality of reference frame translation tools. 
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1 8. The interactive system according to Claim 1 7 further comprising an 
endoscopic viewing system- 
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Attorney Docket No. 5074-00001 3/REA 

REISSUE APPLICATION DECLARATION AND POWER OF 
ATTORNEY 
(BY INVENTOR OR ASSIGNEE) 

DECLARATION BY THE INVENTORS 

As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated 
below next to my name. 

1 believe 1 am the original, first and sole inventor (if only one 
name is listed below) or an original, first and joint inventor (if 
plural names are listed below) of the subject matter that is 
described and claimed in letters patent number 5,868,675, 
granted on February 9, 1999, and for which invention I solicit a 
reissue patent on the invention entitled Interactive System for 
Local Intervention Inside a Non-homogeneous Structure, the 
specification of which 

1 ] is attached hereto. 

[X] was filed on 02/08/2001 as Application Serial No. 09/784,829 
and was amended on 02/08/2001 and as amended hereto . 

I ] 1 hereby declare that there is no assignee for this application. 

ACKNOWLEDGEMENT OF REVIEW OF PAPERS AND DUTY 
OF CANDOR 
(37C.F.R.§1.75) 

1 hereby state that I have reviewed and understand the contents 
of the above identified specification, including the claims, as 
amended by any amendment referred to above. 

1 acknowledge the duty to disclose information which is material 
to the patentability as defined in Title 37, Code of Federal 
Regulations, section 1.56. 

PRIORITY CLAIM 

I hereby claim foreign priority benefits under Title 35, United 
States Code, section 119(a)-(d) of any foreign application(s) for 
patent or inventor's certificate listed below and have also 
identified below any foreign application for patent or inventor's 
certificate having a filing date before that of the application on 
which priority is claimed: 

PRIOR FOREIGN APPUCATiON(S) 

Priority Claim 

8913028 FR 05/10/1989 X 

(Number) (Country) (Day/Month/Year filed) Yes No 



(Number) (Country) (Day/Month/Year filed) Yes No 

1 hereby claim the benefit under Title 35, United States Code, 
119(e) of any United States Provisional application(s) listed 
below: 

PRIOR PROVISIONAL APPLICATIONS 



(application serial number) (Month/Day/Year filed) 



(application serial number) {Month/Day/Year filed) 




DECLARATION ET DELEGATION DE POUVOIR 

N* de dossier de I'avocat. 5074-00001 3/REA 

DEMANDE DE REDEUVRANCE - DECLARATION ET 
DELEGATION DE POUVOIR 
(PAR L'INVENTEUR OU LE CESSIONNAIRE) 

DECLARATION DE L'INVENTEUR/DES INVENTEURS 

En qualite d'inventeur ci-dessous nomme, je declare par les presentes que: 

Mon domicile, mon adresse postale et ma nationalite sont tels qu'indiques 
ci-dessous a la suite de mon nom. Je pense etre le seul auteur original de 
('invention (si un seul nom est indique ci-dessous) ou Tun des auteurs 
originaux conjoints de finvention (si plusieurs noms sont indiques ci- 
dessous) qui est decrite et revendiquee dans le brevet numero 5,868,675, 
delivrd le February 9, 1999, et pour laquelle je demande une redelivrance 
de brevet pour I'invention intitulee Un systeme reciproque pour 
Intervention locale dedans une structure non-homogene dont la 
description 

[ ] est jointe aux presentes. 

[X] a ete deposee le 02/08/2001 en tant que demande N° 09/784,829 et a 
ete amendee le (s*ii y a lieu). 

[ ] Je declare qu'il n'y a pas de cessionnaire pour cette demande. 

DECLARATION D' EX AM EN DES DOCUMENTS ET 
RECONNAISSANCE DU DEVOIR DE VERACITE (37 C.F.R. §1.75) 

Je declare que j'ai examine et que je comprends le contenu de la 
description identifiee ci-dessus, y inclus les revendications, ainsi qu'elle a 
ete amendee par tout amendement ci-dessus mentionne. 

Je reconnais le devoir de divulguer les informations pertinentes concemant 
la brevetabilite telles qu'elles sont definies a la section 1.56 du Trtre 37 du 
Code des reglements federaux. 

REVENDICATION DE PRIORITE 

Je revendique en vertu de la section 119(a)-(d) du Titre 35 du Code 
des Etats-Unis les avantages de la priority a I'etranger de toute 
demande 6trangere de brevet ou de certificat d'auteur d'invention 
figurant dans la liste ci-dessous et j'ai egalement indique ci-dessous 
toutes les demandes etrangeres de brevet ou de certification d'auteur 
d'invention dont la date de depot precede celle de la demande pour 
laquelle je revendique la priorite: 

DEMANDE(S) £TRANG£RE(S) ANT£RIEURE<S) 

Revendication de priorite 

8913028 FR 05/10/1989 

(Numero) (Pays) (Jour/Mois/Ann6e du d6p6t) Oui Non 



(Num6ro) (Pays) (Jour/Mois/AnnSe du depot) Oui Non 

Je revendique, en vertu de la section 119(e) du Trtre 35 du Code des 
ttats-Unis, les avantages lies a toute demande preliminaire faite aux Etats- 
Unis indiquee ci-dessous: 

DEMANDES PREUMINAIRES ANTERIEURES 



(Numero de la demande) (Jour / mois / annee du depot) 



(Numero de la demande) (Jour / mois / annee du depot) 
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STATEMENT OF INOPERATIVENESS OR INVALIDITY OF 
ORIGINAL PATENT (37 C.F.R. §1.175) 

That I believe the original patent to be 

pq partly 

[ ] wholly 

inoperative or invalid by reason of 37 C.F.R. §1.1 75(a)(1)): 
[ ] a defective specification 
[ ] a defective drawing 

pq the patentee claiming more or less than the patentee had a 
right to claim in. the patent. More particularly, I claimed less 
literally than I had a right to claim in the patent; namely, by 
limiting all of the claims of the patent to the recitation of elements 
unnecessary to define the invention in a literal reading of its 
broadest aspects (although not believed to be so limiting under 
the doctrine of equivalents and other legal principals) and 
primarily by reason of the specific wording of Claim 14 so as to 
specifically not claim an endoscopic viewing system. 

That the error(s) listed above and all other errors, which are 
being corrected, up to the time of the filing of this reissue 
declaration arose without any deceptive intention on the part of 
the applicant (37 C.F.R. §1. 175(a)(2)) 

P<] Corroborating affidavits or declarations of others 
accompany this declaration. 

DECLARATION AND POWER OF ATTORNEY 

I hereby declare that all statements made herein of my own 
knowledge are true and that all statements made on information 
and belief are believed to be true; and further that these 
statements were made with the knowledge that willful false 
statements and the like so made are punishable by fine or 
imprisonment/or both, under Section 1001 of Title 18 of the 
United States Code and that such willful false statements may 
jeopardize the validity of the application or any patent issued 
thereon. 

I hereby appoint Stephen J. Foss, Reg. No. 31,251, and each 
principal, attorney of counsel, associate and employee of 
Harness, Dickey & Pierce, P.L.C., who is a registered Patent 
Attorney, my attorney with full power of substitution and 
revocation, to prosecute this application and to transact all 
business in the Patent and Trademark Office connected 
therewith. I request the Patent and Trademark Office to direct all 
correspondence and telephone calls relative to this application to 
Harness, Dickey & Pierce, P.L.C., P. O. Box 828, Bloomfield 
Hills, Michigan 48303 (248) 641-1 600. 



Full name of sole or first Inventor 



Inventor's signature: 

Date: 

Residence: 

Citizenship: 

Post Office Address: 




DECLARATION D'lNOPERABIUTE OU D'lNVAUDITE DU BREVET 
D'ORIGINE (37 C.F.R. §1.175) 

Je declare que je pense que le brevet tforigine est 

[X] partiellement 

[ ] integralement 

inoperant ou invalide, au sens de la section 1.175(a)(1) du Titre 37 du 
C.F.R., en raison: 

[ ] d'une description defectueuse 

[ ] d'un dessin dSfectueux 

[X] du fait que le tttulaire du brevet revendique davantage ou moins qu'il 
n'en a le droit dans le brevet Tout particulierement, j*ai revendique 
litteralement que je n'en avais le droit dans le brevet, a savoir qu'en limitant 
toutes les revendications du brevet a I'enonce d'elements non necessaires 
pour d6finir ("invention en une description Iitt6rale de ses aspects les plus 
larges (mais sans croire que cela constituait de telles limitations selon la 
doctrine de Inequivalence et autres principes juridiques) et principalement 
en raison du libelle particulier de la revendication 14 pour indiquer 
specifiquement que le moyen dlntervention peut consister d'un systeme de 
visualisation endoscopique. 

Je declare que I'erreur/les erreurs indiquees ci-dessus et toutes les autres 
erreurs, qui font I'objet d'une correction, ont 6te commises avant le d6pot 
de la presente demande de redelivrance sans intention de tromper de la 
part du deposant de la demande. (Section 1.175(a)(2) du Titre 37 du 
C.F.R.) 

[X] Des affidavits et declarations d'autres parties corroborant la presente 
declaration sont joints a celle-cL 

DECLARATION ET POUVOIR 

Je declare que toutes les declarations faites dans le present document 
d'apres ma propre connaissance sont veridiques et que toutes les 
declarations faites d'apres des informations et des croyances sont tenues 
pour veridiques; je declare en outre que ces declarations ont ete faites en 
sachant que les declarations sciemment fausses et autres declarations 
analogues sont sanctionnables par des amendes ou des peines de prison 
ou les deux en vertu de la Section 1001 du Title 18 du Code des Etats-Unis 
et que de telles declarations sciemment fausses peuvent remettre en 
cause la validite de la demande de brevet ou de tout brevet delivre sur 
cette base. 

Je nomme Stephen J. Foss, N° d'enregistrement 31,251, et chaque 
partenaire, conseil juridique, associe et employe de Harness, Dickey & 
Pierce, P.L.C., qui est un avocat en brevets inscrit, pour agir en tant que 
mon fonde de pouvoir ayant pleins pouvoirs de substitution et de 
revocation, aux fins de poursuivre la presente demande et de traiter de 
toutes affaires en rapport avec celle-ci avec le Bureau des brevets et des 
marques de commerce [Patent and Trademark Office]. Je demande au 
Bureau des brevets et des marques de commerce d'adresser toute la 
correspondance et tous les appels telephoniques ayant trait a la presente 
demande a Harness, Dickey & Pierce, P.LC, P.O. Box 828, Bloomfield 
Hills, Michigan 48303 (248) 641-1 600. 

Norn complet du seul inventeur ou de I'inventeur original : 

Jean Francois UHL 

Signature de I'inventeur 

Date: 

Domicile: __ 

Nationaiite: 

Adresse postal e : 
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Full name of second joint inventor: 



Inventor's signature: 
Date: 



Residence: _ 
Citizenship:. 



Post Office Address: . 



Full name of third Joint inventor: 



Inventor's signature: 
Date: 



Residence: . 
Citizenship:, 



Post Office Address: . 



Full name of fourth joint inventor: 



Inventor's signature: 
Date: 



Residence: _ 
Citizenships 



Post Office Address: 



Nom complet du second inventeur conjoint, le cas ech§ant 

Joel Henroin 

Signature du second inventeur 

Date: 



Domicile:. 



Nationality 



Adresse postale : . 



Nom complet du troisieme Inventeur conjoint, le cas echeant: 

Michel Scriban 

Signature du troisieme inventeur 

Date: 



Domicile:. 



Nationality 



Adresse postale : . 



Nom complet du quatrieme Inventeur conjoint, ie cas echeant: 

Jean-Baptiste Thiebaut 

Signature du quatrieme inventeur 

Date: 



Domicile:, 



Nationality 



Adresse postale : 
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